In general, it is desirable for images or a sequence of images (hereinafter referred to as a “video sequence”) captured using an image capture device to be processed, either during capture, or when viewed or played back using video equipment. The processing will generally take the form of filtering or correction of the image or video sequence in order to remove undesirable elements such as motion blur, for example, which may be caused by movement of the image capture device during the capture procedure.
Such processing is termed ‘image stabilization’, and is the process which allows an image or video sequence to be captured, stored or rendered with a reduced (or eliminated) amount of apparent motion caused by the secondary, unintentional motion of the image or video sequence capture device with respect to the scene or object being captured, whilst preserving the dominant, intentional motion of the capture device. The image or video capture device could be a camera (digital or other), or a camcorder (digital or other), or generally, any device capable of capturing an image or sequence of images for storage in the device or elsewhere.
Several classes of image stabilization methods exist, viz. mechanical, electromechanical, optical and electronic. The above stabilization methods are well established and documented, and will not, therefore, be described in any further detail.
Recently, interest in the use of user-wearable image capture devices has resulted in the above stabilization methods being applied to the images captured therefrom. Such a user-wearable device may have stabilization functionality built-in, or may rely on stabilization of captured image data being carried out in another system distinct from the device itself.
The nature of a user-wearable image capture device implies that any images captured from it will be intrinsically ‘noisy’ due to motion and vibrations introduced to the device (and hence images captured therefrom) by a user's movements, intentional or unintentional. Such ‘noisy’ image data may best be characterized as data which includes a secondary (to the data of the scene or object to be captured), undesirable element such as a low-frequency element in a video sequence due to a user walking or running, for example, or a high-frequency element in a video sequence due to a user travelling in a vehicle, for example. It will be appreciated that further types of undesirable data may be introduced into image data due to device rotation etc.
In order to reduce the problem of ‘noisy’ image data, the above-mentioned stabilization methods have been applied to captured sequences. For example, U.S. Pat. No. 5,502,484 describes a shake detection and correction method for use in video cameras and video signal reproducing apparatus. Further, U.S. Pat. No. 5,253,071 describes a method and apparatus for stabilizing an image produced in a video camera.
A problem associated with the above stabilization techniques, especially when applied to user-wearable image capture devices or portable capture devices where a video sequence is to be captured, is that any intentional movement data which forms part of the video sequence data to be stabilized will be filtered out in the process of image stabilization, especially since the amplitudes of motions associated with such intentional movements are generally smaller than the induced unintentional ‘background’ motion of the capture apparatus.
In U.S. Pat. No. 5,253,071, the start and stop of intentional panning or tilting is detected. A vibration detection signal is cancelled during a period between detection of the start of intentional panning or tilting and detection of the stop of the intentional panning or tilting. Thus, stabilization is switched off when intentional motion is detected.
The preservation of intentional movement data is important since, during consumption for example, such intentional movements allow someone viewing a video sequence to be able to ‘re-live’ a situation as experienced by the user of the device.